skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Search for: All records

Creators/Authors contains: "Qiu, Ruo_Ning Nancy"

Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

  1. Producing open-ended creative work through crowdsourcing remains a challenge, as workers often lack domain expertise, and requesters struggle to provide scalable guidance. Can the workers themselves create materials that guide subsequent workers? In this paper, we prototype a workflow for emergent scaffolding, where hints, rubrics, and examples are generated by crowd workers after attempting the task. We demonstrate how an iterative Train-Try-Reflect-Synthesize pattern—supported by LLMs—can produce a structured rubric with graded examples to guide subsequent workers on a task to create digital illustrations for scientific papers. To evaluate this strategy, we conducted a between-subjects experiment with three conditions: baseline instructions, generic examples, and emergent scaffolding. Participants in the emergent scaffolding condition created significantly better illustrations, as rated by blind-to-condition judges, compared to generic examples or instructions only. While self-efficacy ratings were mixed across conditions, emergent scaffolding participants provided better feedback during their post-task reflections. We discuss the potential for emergent scaffolding to support and scale up complex, creative tasks in crowdwork. 
    more » « less
    Free, publicly-accessible full text available August 3, 2026